Mctf and Scalability Extension of H . 264 / Avc

The extension of H.264/AVC hybrid video coding towards motion-compensated temporal filtering (MCTF) and scal-ability is presented. Utilizing the lifting approach to implement MCTF, the motion compensation features of H.264/AVC can be re-used for the MCTF prediction step and extended in a straightforward way for the MCTF update step. The MCTF extension of H.264/AVC is also incorporated into a video codec that provides SNR, spatial, and (similar to hybrid video coding) temporal scalability. The paper provides a description of these techniques and presents experimental results that validate their efficiency. 1. INTRODUCTION In recent years, various approaches to MCTF-based video coding have been presented (e.g. see [1][2]). The main reason for the advances in coding efficiency of these codecs is the possibility to use techniques for motion compensation (MC) known from hybrid video coding through the lifting representation of filter banks [3]. These filter banks are typically cascaded sequences of (motion-compensated) prediction and (motion-compensated) update steps. Because lifting is invertible, any MC technique can be incorporated into the prediction and update steps of the filter bank. By using the highly efficient motion model of H.264/AVC [4][5] in connection with a block-adaptive switching between the Haar and the 5/3 spline wavelet, both the prediction and the update step are similar to MC techniques in generalized B slices of H.264/AVC. Furthermore, the open-loop structure of a temporal sub-band representation offers the possibility to efficiently incorporate SNR and spatial scalability. SNR scalability is achieved by residual quantization with very little changes to H.264/AVC. For spatial scalability, a combination of MCTF and over-sampled pyramid decomposition is proposed, which requires some additional mechanisms to convey bit rate from lower resolution to higher resolution layers. However , for each layer, the macroblock-based structure of H.264/AVC can be maintained as will be shown. Because of the similarities in MC, the approach to temporal scalability of H.264/AVC is maintained. Motivated by these facts, we have investigated the possibility of a simple but yet efficient extension of H.264/AVC hybrid video coding towards MCTF and scalability. The next section describes the MCTF approach. Section 3 shows how H.264/AVC is extended towards MCTF. Section 4 describes the scalability extensions. Section 5 provides experimental results.